DOCOMEMT ItESOHE 



FO 096 352 



TH 003 968 



AHTHOF 
.TTTIE 

INSTITUTION 

SPONS AGENCY 

PUB t>ATP 
KOTB 

S^SS PRICE 
DESCRIPTORS 



Goodvin, HilliaD L.', Cosp. 

Evaluation InstruBents for Fesearch Training 
Prograis. 

Coloradb Univ., Boulder. Lab. of Educational 
Research. 

National Center for Educational Research and 
Develt>'pBent (DHEfT/OS) , Washington, D.C. 
£73] 

56p,; For related docuients, .see TB 0.03 967-973 



«F-$0.75 HC-$3. 15 PLUS POSTAGE , 
boctosral Prograa^; '('Educational Researchers; 
*6rfii,si,uate Students^ Internship Prograiis;Ntfinterviews; 
♦Questionnaires; *Tests; JPraining Laboratories 



ABSTRACT 

_ ThiS" (iocuaeat contains instruaents used in' the yearly 

evaluatdons c** 1%^ Laboratory for Educational Research (LER) of the ^ 
University ot Colorado between 1966 and 1970, It also includes the 
instxuaents used th:» suBBer 1973 LER Susaative Evaluation. The . 
purpose of the eva^tUitivn vij$ to determine the effectiveness of thi$ 
research trjiining. proc;r?B. according to the perceptions of different 
audiences. The j.nstrcBents included questionnaiifes and interviews 
fo*os vhieh irer« adaiiiistersd to past and 'present students of the LEE 
prograa, f acuity , bf^ the school of education, faculty of LER, and 
clients of LEP. (Author/SE) 
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EVALUATION INSTRUMENTS 



FOR RESEARCH TRAINING PROGRAMS 



^ BESF cow AVAIU^LE 

NCERD Reporting Form — Develppmenral Products 





• 

t. NoM* of Product 

Evaluatlo^t Instruments for 
Research Training Programs 


2. ioborcitory or CtnNr 

Laboratory of Educational 
Research, University of 
Colorado 


3^ Ktpoft Prtporatton 

DaUnrerkired 11/30/73 
Re\Ammd bu^*^^ GoOdwin 


9 


4. frobloMt Description of the edueatUnal prcblm thin product de$igned to 3i>lv<s. 

A requirement to evaluate the effectiveness of research training programs 
according to the perceptions of different audiences. 

• 


S. Sfrotagyi fhe general atrat^j/j^ 90leffted fpr'tlw wlution of th0 problem above. 

1. '.Evaluation instruments were developed and utilized by LER staff and fellows. 

1 

2. .'When and if reused, particular items were subject to revision. 

i ■ ■ 

• 

' ' 1 

• 

t 

' / 


Mitcut Dafttt Apprcxirtate date 
prc^duujt MS (or be) reid}^ 
for release to next agencu. 

December, 1973 

• 




istic level (or rroj-^i'tcci levoZ) 
of developnent jf praductt at Time 
of release^ C-xeak one^ , » 
X Ready for critical rerrCev and for 
px^paration for Field Te$c ,^ 
fi.e. prototype 'ndterialsJ^ 
/Ready for Field Te3t 
Ready for publieher modification 
Ready for general H^aerrrLnation/ 
diffu3i(yK 


8. Hmnf Ag«ncyi ^j^nc% to uhoK 
product vjj (:r 'Hit te) 
retcaQcd for furt her 
icvelop*^c*it diff.tsicK. 

NIE 
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BEST COPY AVAILABU 



9. Product D«icrlptt0fii D0$cribc the follouing: mmAer each dm»ortption. 

• ChaiMctcrif ticti of the praduet. Aeeooiated products^ if any. 

# Km it Mrhf. * S« Special conditions, time^ training. 



1. A series of evaluation Instruments. 

2. Administered to selected audiences for written reaction to or. In soine 
cases, oral reaction via structured iriterview. 

3. Provide data bearing on the effectiveness of the research training 
program. 

4. None. ' 

5. In their present form, these are very rough instruments with no available 
supporting data bearing. on validity or reliability. 
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BEST COPY A)fAfU\BI£ 



to. Product Ut^rii ThoB^ in4ividual9 or group& expected to me the produat. 

Agencies 'and 1nst1tult&Q|^ engaged In research and evaluation training. 

II. Product Outtem^ti The changes in user inthainort atHtudeSt efficiemtt, litcr. result ittit 
fr^m pr^uct utOt ao surporteU data. PZaoM oitm T0t»uant aupport duountenta. tf 
clMJnjt for the pedSuot sre r^" jet 'supported /«- ampiriaal ai>id0n<ie pLtiiat .'nUf i'. 



1. See previous evaluations filed with USOE on thp effectiveness of the LER 
training program. 

© • 

2. Additionally, see the Laboratory of Educational Research (LER) 1966-1973 
report released December, 1973, in which certain of these evaluation 
instruments were utilized. 



12. Pet»nfiol Edueatiodol CoRs»qv«nc*st Discuss r:ot only tfec theoretical (i.e. oomeivable ) 
implications oj' your product but also the more vrobahlfi inpLications of vour prodmtt 
ospeciall'j over the next decade. 

Improved effectiveness of research and evaluation training activities. 
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13. rrodwet El*««(itti 

^ List the elenanta nhich oonatitute the produot* 


14. OHsitti 

Circle the met 
appropriate letter. 


Evaluation Instruments. 


(d)n a 




DMA 




a M A' 




DMA 




D M cA 




DMA o 


♦ 


0 M A 




D H A 




D H A 




DMA 




*D M.A 




DMA 




DMA' 




/ ■ ■ 


DMA 






DMA 






£N» Developed 
tf^ Modified 
A» Adopted 


!$• Stort^yp Co>tftt Total oxpeated co&ta to procure^ 
install and initiate use of the product. 

Variable but quite minimal. 


• 

16. Op«roting Costti Projected costs for continuing 
' use of product after initial cuioption and 
installation (i.i^^fees, consumable supplies^ 
special staff, training, etc.). 

See 15. 


17. Lik«ly Mork^h (/hat ia the likely market for thia product? Consider the size ami type of 
the user group; rtzdmber of podsible siibstitute (cof^etitorJ products on the market; and 
the likely opailability of funds to purchase product by^ (for) the product user group. 

y Uncertain. 
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DOCUMEtiT 1 



Instruments Used in the 
1966-67 LER Evaluation Report 



EVALUATION QUESTIONNAIRE FOR LER STUDENTS: -1966-67 



I. Evaluation of the Core Curriculum 

Your evaluation of the core courses that you have completed will provide the 
stimulus and support far proposed revisions. Please be candid. 

Make your evaluation in terms of the objectives of the Graduate Research 

Training Program. Place your rating to the left of the cdurse number on the 

adjacent form. 

The course is: > 
Rating Code: 

A: essential 

B: highly desirable 

C: desirable, but other not-required courses ivould be equal in value 
D: not as valuable as other possiblo electives 
E: of little or no value 

I: poor rating resulting from the quality of the instruction rather 
than the course content ger se. 

The required courses are listed below along with the student evaluations. 
It is important to bear in mind that these courses were evaluated with respect 
to their contribution to the objectives of the Graduate Research Training 
Program and not in relation to the students' own objectives or those of other 
students in the University. 



t 



Education 

480-3 Elementary Statistical Methods 

503-2 Advanced Psychological Foundations 

505-3 Intermediate Statistical Methods 

511-3 Advanced Educ. Measurement & Eval. 

516-2 Advanced See. Found, of Educ. 

591 -3 Eval . of School Systems &* Programs 

600-2 Methods of Educational Research 

604- 3 Exp. Design and Analysis I « 

605- 3 Exp. Design and Analysis II 

608- 0 Internship in Educ. Research I 

609- 0 Internship in Educ. Research II 

610- 0 Internship in Educ. Research III 

611- 0 Internship in Educ. Research IV 
700-4 Masters Thesis 

Psychology 

587- 4 General Statistics 

588- 4 General Statistics 

654-2 Meth. in Assessment of Soc. Structures 

691-3 Multivariate Analysis 
Mathematics ' 

205- 3 Math, for Social Scientists I 

206- 3 Math, for Social Scientists II 
Computing Science 

551-3 Computer Applic. in Behavioral Sci . 



Ratings 
A B C D E I 



A B C D E I 
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Sociology 

505- 2 Proseminar In Sociology I 

506- 2 Proseminar In Sociology 11^ 

V 

507- 2 Seminar In Research r^thods I 
Philosophy 

542-3 Philosophy of Science 



(: 



SA A ? D SU Comments 



II. Evaluation of the Graduate Research Training Program 

Please respond to the following statements , selecting one response -based on 
the follqvfing key: SA (Strongly Agree) , ^ (Agree), ? (Undecided) , D (Disagree), 
SD (Strongly Disagree). 

1. The objectives of this program are clear to me. 

2. The objectives of; this program are not realistic. 

3. The participants accept Ithe purposes of this 
program. \ 

4. The objectives of this program are not the same 
as my objective^. 

5. r have not learned much new since being In- the 
program. 

6. The program has made possible a degrer>pf 
research competence that would not have been 
otherwise possible for me. 

7. The program has had little effect on other 
graduate students in the School of Education. 

8. We work together well as a group. 

9. The informal group interaction Is valuable. 

10. There is little time for informal conversation. 

11. My time is being well spent. 

12. The program is meeting my expectations. 

13. Too much time is devoted to trivial matters. 

14. The program should be more flexible to meet 
individual needs. 

15. The research training core courses are relevant 
to the program's objectives. 

16. The importance that I attach to research and 
evaluation in education has greatly Increased. 



17. The role of research in my ultimate 

professional position will be increased 
markedly by this program. 

18.. The internship experiences in the public 
schools are very valuable. 

19. Too much internship time is required 
for meeting the relevant objectives. 



SA A ? D SD Comments 
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III. Evaluation of LER In General 



For each of the seven concepts below, mark one number (from 1 to 7) 
indicating your attitude toward the concept. 
Concept : LER Fellows 



Bright 
Poorly motivated 
Flexible 
Young 



1 2 3 4 5 6 7 

1 2 3 4 5 6 7 

1 2 3 4 5 6 7 

1 2 3 4 5 6 7 



Dull 

Highly Motivated 
Infl exible 
0]d 



2. Concept : Physical Facilities (Rooms, Desks, etc.) 



Inadequate 
Convenient 
Above Average 

^* Concept ; LER Directors 

Approachable 
Not Helpful 
Scholarly 
Infl exible 
Competent 

4. Concept : Internship 



1 



Valuable 
Too Much ^ 1 
Essential 1 



2 3 4 5 6 7 Adequate 
2 3 4 5 6 7 Inconvenient 
2 3 4 5 6 7 Below Average 



2 3 4 5 6 7 Unapproachable 

2 3 4 5 6 7 Helpful 

2 3 4 5 6 7 Not Scholarly 

2 3 4 5 6 7 Flexible 

2 3 4 5 6 7 Incompent- 



2 3 4 5 6 7 Worthless 
2 3 4 5 5 7 Too Little 
2 3 4 5 6 7 Not Essential 



Concept ; Financial Support (Fellowship Stipends) 

Too Little 1 2 3 4 5 6 7 Too. Mych 
Fair ' 1 2 ' 3 4 5 6 7 Unfair 

6. Concept : LER Clients (Students and Faculty Served by LER) 



Well Prepared 
They Benefit 
Ungrateful 



2 3 4 5 6 7 
2 3 4 5 6 7 
2 3 4 5 6 7 



7. Concept : LER Core Curriculum 

Too Much' Math 1 2 3 4 5 6 7 

Too Theoretical 1 2 3 4 5 6 7 

Poorly Designed 1 2 3 4 5 6 7 

•Too Little Freedom 1 2 3 4 5 6 7 



Poorly Prepared 
They Don't Benefit 
Grateful 



Too Little Math 
Too Applied 
Well Designed 
Too Much freedom 



Rank order - from "1" highest to "6" lowest - the following oost-doctoral 
activities on the basis of your own preference: 

Research in an R & D/Center. 

Research in. a Regional Laboratory. 

Director of research in a large city school system. 

'• Research and teaching in a small (less than 4,000) college or university. 

Research and teaching in a large (10,000+) unive*'sity. 

Teaching only in a university. 



Rank order - from "1" highest to "6" lowest - the following post-doctoral 
activities in terms of what you feel the LER Directors would like you to be 
doing when you finish. 

Research in an R&D Center. 

Research in a Regional Laboratory. 

Director of research In a large city school system. 

Research and teaching in a small (less than 4,000) college or university. 

Research and teaching in a large (10,000+) university. 

Teaching only in a university. 

Mst the professional education orjocial science associations of which you 
uscarae a member during the '6&-Wircademic year: 



Check the professional journals and periodicals which you read regularly: 

Anerican Educational Research Journal 

Educational & Psychological Measurement 

Journal of Educational Measurement 

Journal of Educational Psychology 

Journal of Educational Research 

Phi Delta Kappan 

' Psychological Bulletin 

Review of Educational Research 

Personnel and Guidance Journal 
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EVALUATION QUESTIONfJAIRE FOR 
SCHOOL OF EDUCATION FACULTY 
ON THE LABORATORY OF EDUCATIONAL RESEARCH 

1966-1967 



-We are gathering information to evaluate the operations of the Laboratory 
of Educational Research during the '66- '67 academic year (including summer 
sessions) and it^ order to plan for the current year. Your responses to the 
following questions will be appreciated. If you are new on the faculty, 

» 

undoubtedly you will not be able to respond to these questions. When you have 
completed this questionnaire, please return it unsigned to the mailbox of either 
Gene Glass or Ken Hopkins. Thank you. 

1. a. Have you personally used the services or facilities qf the Laboratory 
of Educational Research (LER)? 

* Yes 
No 

b. If "Yes," how often? 

c. If "Yes," for what purpose or in what manner? 



2. a. How many of your advisees used the services or facilities of LER? 



b. If one or more of your advisees did use LER services, in what manner 
or for what purpose were they used? 

/ 

3. a. Did you attend any special presentations or meetings erfi^anged and 
conducted by LER during the '66- '67 academic year? 

Yes 
No 



b. If "Yes," approximately how many? 



'4. Do you feel that the activities and objectiv.es of LER are helping to meet 
the research needs of the School of Education as identified in the report 
of the Ad Hoc Conmittee on Research, Measurement, and Statistics aid the 
1955 School of Education Sel f -Study: 

Yes 
No 



The LER trainees for the. period indicated were as follows: 

Glenn Bracht Ralph Hakstian 

Ann Brickner Lyle Knudson • 

Russell Chadbourn Percy Peckham 

Scott Harrington Robert Smith 

Please indicate Strpng Agreement (SA), Agreement (A). Indecision (?). Disagreement 
(0), or Strong Disagreement (SD) to each of the followinq statements as they apply 
to the above LER trainees. (If you know none of these students, omit US - #8). 



SA A ? D SD 



5. They seem not to have an adequate interest in 
education. 

6. They do not seem to be willing to give assistance 
to other gradu^e students on research and evalua- 

^ tlbn problem^ 



7. They are primanly interested in statistics (as 
opposed to testing, educ. psych., counseling, 

math educ, etc). >^ 

8. They seem to have the qualities needed to bp 
effective as: 

a. university professors 

b. public sch^l research workers 

Please mark either SA, ,A, ?, D, or SD for each of the following statements as 
-they pertain to the operations of LER. 



SA A ? D SD 



9. LER should tutor Education graduate students 
having difficulty in statistics courses. 

10. LER has been successful in improving the quality 
of research performed in the School of Education 
a. by graduate students 



c 



b. by faculty ] . 



V A ? D SD 

n. LER is not being of service to the public 
schools of Colorado. 

12. In training graduate students, LER is placing 
too much emphasis on research and not enough 
on learning about other areas of education. 

13. If I had a question pertaining to research | 
design, measurement, statistics, or computer 
use, I would be reluctant to consult LER for 
assistance. 

14j Please sug^st ways in which LER could be of greater service to the research 
efforts of the entire School of Education (including faculty, graduate 
students, etc.) : / 



\ 
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DOCUMENT 2 



Instruments Used in the 
1968-69 LER Evaluation Report 
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Questionnaire on Laboratory of Kducational Research 
Concept: LER Fellows 



iiright 1 

Poorly Motivated 1 

Flexible I 

Ambitio\is 1 

Young 1 



2 

2 

2 
2 



3 4 5 6 7 null 

3 4 5 6 7 fllRhly Motivatftd 

3 4 5 6 7 Inflexible 

3 4 5 6 7 Lazy 

3 4 5 6 7 Old 



Concopt ; Physical Facilities a) (For personal academic work) 



Inadequate I 

Convenient 1 

Above Average 1 

Inadequate 1 

Convenient 1 

Above Average 1 



2 
2 
2 

2 
2 
2 



3 
3 
3 

3 
3 
3 



4 
4 
4 



f 

5 
5 



6 7 Adequate 

6 7 Inconvenient 

6 7 Below Ave rape 



b) (For consulting asslfmments) 
4 5 6 7 Adequate 
4 5.6 7 Inconvenient 
4 5 6 7 Below Averape 



Concept ; Equipment for Data Anal/sis 

Inadequate 1 2 3 4 5 6 

Convenient 12 3 4 5 6 

Above Average 1 2 3 4 5 6 



Concept : Library of Reference Materials 



Inadequate 12 3 

Convenient -J. 2 3 

Above Averajyo 1 2 3 

Concept ; LKR Directors 

Approachable 12 3 

N'ot Helpful 1 2 3 

Scholarly 1 2 3 
Competent 

Administrators 12 3 
Respected 

Personally 12 3 

Concept : Fellowship Stipends 

Too Little 1 2 3 

Fair 1 2 3 



4 
4 
4 



4 
4 
4 

4 

4 



4 
4 



5 
5 
5 



5 
5 
5 

5 

5 



5 
5 



6 
6 
6 



6 
6 

6 

6 

6 



6 
6 



7 Adequate 

7 Inconvenient 

7 Below Average 



7 Adequate 

7 Inconvenient 

7 ^elo'.; Ayeraf»e 



7 •'lot Anproachable*' ' 

7 Helpful 

7 vot Scholarly 

7 Incompetent Administrators 

7 '''ct Respected Personallv 



7 Too., ^-fuch 
7 Unfair 



Concept : Financial Support (Professional activities o.p, travel to 
conventions) 



Too Little 12 3 4 5 6 7 Too Much 
Fair 1 2 3 4 5 6 7 Unfair 



Concept ; IXW Core Curriculun 



Too "<uch Math 1 

Too Tlieoretical 1 

Poorly r)osis>ncd I 

Too Little Freedom 1 



3 

3 

3 
1 



4 5 

4 5 

4 ^ 

4 3 



7 Too \At t^- ^ » 
7 Too An^^l 



Too ""'nci'. ^roc^Hi»t 



^ on cent : Si'rrinar rpoal c-r*^. (Trn^: ,u^u-r .c*;'.>r1'<, ."tn>u-. c^t.s,) 

: i 4 ^ - 

: 1 A ^ : 7 

2 ^ sr.? 



Tnt> Often 



\\ foiu-ept : L.'^: ^Mionts (Students .^no ^'iculr^- Sf-rvt' T/'^) 

r.iov '.onoflt 1 ? 3 /♦ 
rn^.ratoful 12 3 4 

! pnjcp pt: Internship :n!porvlsion 



5 



r. 



7 



^^rntoful 



Too Closely 
Sunervisec! 
\s <; 1 i*n re:i t ^ ('1 oar 



Ho/-,.!-" TIno ClviMi for "n^^V.s 



* L !* ;'WMir i ,i ! 



The list below contains some of the kinds of activities and tasks that I*F^ 
^Vllovs mis^ht purfonn as part of thelj- intern ?;hlp dutios. Pleasn respond 
to each item 'n the list In th*^ two \^ays listed helox;? 



FIRST 

Chock frequency of 
occurrence for each 
activity 



Check appropriate value you attach to each task, 
whether or not you vere called upon to perform ft< 



Oltan 


Sometimes 


Sdldom 


Never 
































































































































































































































































































1 f 






: \ 1 




r ' 














1 

i Often i Sometimes 
L_ i 


Seldom 


Never 



Led Student IVS Oiscusslon (Rrouni Sack, 
S|>ecial Seminar) ^ ^ ^ ^ 

Helped with ceseaix^h budgets ] * 

Assisted In administrative duties 

Took minutes in meetings 

Planned and arranged aeminars or? 

Attended professional roeetings . • * [ [ 

Assisted with supervisor's profession*! correspondence! ! 

Helped draft or edit supervisor's publications 

Filled out questionnaires for supervisor received from' 

outj>ide of Lah. 
Designed a reseeux^h study . . • ♦ , 

Reviewed and/or abstracted literature on a topic ! ! ! ! ! 

. ConceptuaUzed a research problem [ ] [ 

Assisted in other conceptual activities 

Wrote a research proposal «^ * 

Designed statistical analysis 

Wrote computer programs 

Conducted Experiments 

Constructed research instruments [ [ [ 

Administered or scored research instniments !!!!!!!! 

Collected data in other ways * 

Assisted in actual teaching p£ courses! !!!!!!!!!! 

Read or graded papers ••.!!!!!!** 

Administered or scored classroom exams 

Assisted in preparing lectures or speeches for supervisor ! 

Did typing, filing, and/or answering telephones 

Did collating, duplicating, tallying, and/or bookkeeping ! 

Used calculator for data analysis 

Made tables, graphs, or charts . • . ; ! ! 

Coded and/or tabulated data •••.♦•••!!!!!!!! 
Keypunched and/or verified data •••••!!!!!!!!! 

Submitted data for computer analysis 

Used other computer-related equipment 

Interpreted data !!!!!! 

Helped write up final research report !.!!!! 

Wrote research article • • • ♦ ! ! ! ! 

Gave a research ^per !!!!!! 

Set up physical facilities for conferences or lectures ! ! ! 

Assisted supervisor with personal matters ! . 

Engaged in manual labor (moving furniture, etc.) 
Gathered data for thesis or dissertation • . . ♦ 



Hao 
3reat 
value 


Had 
Soiw 
Value 


Had ' 
Little 

or no 
^'alue 


















































- L 




\ 
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/■ 






1 






» 

-f 






i 




j 


f 










































































































Racf "1 
:;reat 


Had 
Some 

Vr»lue 


lad 

Little 
^r tio 

Value 



o 

ERIC 



14. Name the tlirce persons wlio^ in your opinion, appeared to xnahe two greatest 
pro^'.ress toward the objectives of t!ii?5 program durinr '^P.-'ft ^: 



Not hi^ ported-^- ?Tost progress: 

^r^latemal Use ^^^^ progress: 



Tliird most progress: 



n, N'arv the thrco persons wUo, in your opinion, appf^arod to attain the highest 
level of competence by the end of *6^~*fi'): 

Not Reported- iiijvUesf 

For Internal Use uiguest. _ 



Only s'ext highest: 

"Hiird hlp.hest 



ERIC 



10. Rank order— from 'T' highest to "6'* lov^ej^t— the followinR post-doctoral 
activities on the basis of your n\m proferrnce: 

Rank Activity 

^xcsearch in an n «Si n Center. 

Hesearch in a TveRional Laboratory. 

nirector of research in a large city school sy??tem. 

Research and toachinj^ in a nrnall (less than 4,000) col lepe 

or university. 

research and tcnchin;; Ln a Jnrpc (1^,000+) university. 

Teachinp only in a unlvrr*^itv. 



17. f\in!; order — fro!P, "T' hirjiest to ''f** loi;est — the follovinp. post-doctoral 
activities in terms of what you feel the hKH T)i rectors would like you to 
!)e doin<: xchen you finisli: 

Hank Activity 

Research in an ^ St \) Center 

Pxesearch in a Repional Laboratory. 

.Director of research in a larpc citv school systep. 

Research and teachinp In a small (less than 4^000) coUepo 

or university. 

Rc?^carch and teaching In a larpe (10^000+) university. 

Teachinp only in a university. 



18. List any professional education or social science associations of which vou 
became a member during the academic year: 



10. Chfck the professional journals* and periodicals which vou n»ad reiiularlv: 

American Kducational Resoarcli Journal 

' Biometrics 

ISioTHOtrika 

T lilducational vSt PsvQh. >?eas* 

Harvard Educational Review 

Journal of Applied Psycholog^^ 

Journal of education ^ 

Journal of Educational* Mcasurf^men t 

Joum-»l of educational Vsyc5iolo}7v 

Jouraal of Educational Kesearcli 

Journal of Erperlmontal Education 

Phi I)elta Kappan 

Psvcholofical [Jul letin 

Psvcholoe;ical Review 

Psychomctrika 

; Review of Kducational research 

Teachers Collepe Pecord 

Othnrs (??noci fv) 



erIc 



/ 

I 



t 



A.najor concern at thp llKR is to find ways of Imnrovinp the preparation of 
c?ducational researchers throunh optimizing experiences provided to fellovs. 
We would appreciate your frank and serious suRpestions repardinp, ways to 
improve the LER program. We are interested both in suggestions for new 
procedures and critiques of exj.stinr procedures. Please use the space 
helou for your 'response. 




\ 



.DOCUMENT J 

Instruments Used in the 
1969-70 LER Evaluation Report 
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QUESTION SETS USED WITH LER PRINCIPAL STAFF; FALL, 1973 



A-1 : Over the seven-year history of LER, there have been several drop-outs; Is 
it possible for you to characterize the drop-outs as a group (i.e., did 
they have coawion characteristics)? 

A-2 : For several of the early years of the Lab, monies were received to develop 
the research methodology program. Comment upon the present pIFbgram as 
compared to the situation existing before LER was initiated. 

A-3 : Has the effort you have expended in developing and operating the LER model 
been worth it or not? Elaborate. 

A-4 : Rank order the importance or weight of the following LER entrance criteria. 
(Give a rank of 1 to the most important, etc.) 

1) GPA 

2) GRE 

3) Math background 

4) Commitment to educational research 

5) Endorsement from known, reliable source 



A-5 : Do the high entrance criterion for LER students have any undesirable 
consequences? 

A-6 : To what extent do you feel the students of the School of Education have 
been influenced by LER? Be specific in detailing examples of this 
Influence. 

A-7 : To what extent do you feel the faculty and students in departments other 
than education have been i^ifluenccd by LER? Be'^specific in detailing 
examples of this influence. 

A-8 : Comment on how you would view the following in terms of appropriateness 
for the LER training model : 

1) a large scale research project in which students could be actively 
Involved. 

2) increased internships with "applied" agencies, that is, public schools, 
state departments, and the like. 

A-9 : Should LER students be required to do a small research project from 
conceptualization to completion? 

A-10: How important is it that LER fellows teach while earning the Ph.D.? 

A-11 : Do you view the LER student of today, as compared with 1966, as taking 
more or fewer offerings in the various behavioral sciences? 

A-12: At one pointy internships were reduced from 600 to 400 hours. What do 
you view as the benefits and consequences of this change? 
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A-T3: Have you noticed a shift In where LER graduates are employed? 

A-14: Assuming th t the current trend toward difficulty in finding student 

support funus continues, what effect do you predict this will have on the 
LER program and students? 

A-15: To what extent do you feel that LER students are handicapped by their lack 
of concentration in a substantive area (i.e., an area other than research 
methodology itself)? 

A-16: To what extent are LER students in touch with the major problematic areas 

in education today (i.e., integration, alternative schooling, accountability, 
etc.)? 

A-17: In your opinion, and considering similar programs to LET? across the country, 
how would you rank the LER program in terms of Its contribution to the 
field of educational research? 

A-18: To what extent do you feel that LER graduates have or will become leaders 
in the field of educational research, broadly conceived? (Be specific, 
that Is, how many do feel will attain what level of prominence?) 

A-19: What do you perceive as the greatest weakness of LER? The second gieatest 
weakness? 
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B-1 : At the outset of LER in 1966, it was envisioned that several departments, 
other than education, would be meaningfully involved in the LER student's 
program. c:To what extent do you feel such inter-disciplinary liaison has 
been achieved and also cbrroent on the value of such liaison? 

t' 

0-2 : Early in the life of LER, it was anticipated that there would be ?4 

students training at a single time in the Lab. If such a number had been 
attained, what do you view as the likely benefits and consequences? 

B-3 : To what extent do you feel the success of the LER program has been depen- 
dent upon recruitment? Upon high entrance standards? 

B-4 : To '-hat extent do you -see LER as betng vulnerable to the charge of academic 
"elUism"? What positive aspects has such "elitism"? What negative 
aspects?^ 

B-5 : Speculate on how successful LER students would have been after completing 
the Ph.D. if they had completed it in an area other than LER. ♦ 

B-6 : To what extent do you feel that the internships (608, 609, 610, 611) are 
critical for the LER program? 

B-7 : To what extent would you estimate LER students are involved in the following 
activities via their internships? Estimate for both early Lab (say, 1966- 
r>8) and late Lab (say, 1971-73). 

1) Consulting with graduate students. 

2) Consulting with faculty. 

3) Running data analysis computer routines. 

B-8 : Coiment upon the nature of your personal involvement with your students, 
that is, its duration, its intensity, its merit, etc., particularly as 
compared to involvement with students displayed by other faculty. 

B-9 : Trace the history of the brown sacks. Have they recently been what you 
want them to be? 

t 

B-10: To what extent do you feel the faculty of the School of Education have 
been influenced by LER? Be specific in detailing examples of this 
influence. 

# 

B-11: At times, the School of Education faculty have expressed concern about the 
accessibility of LER given its location in the Education Annex. Some have 
suggested that it be moved to Hell ems Annex. What is your reaction to 
this suggestion? 

B-12: Do you notice a trend in terms of the number of LER students who leave the 
program for employment before completing their dissertation? 

B-13; To what extent do you perceive major changes in the competencies e quired 
by a LER student as he completes the Ph.D.? 
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B-14: Several students in reflection feel they would have gained vta more time 
(i.e:, another year) in LER. How do you feel? 

B-15: Should LER students be involved in preparation of research proposals for 
funding? 

B-16: To what extent do you feel that it is important for LER students to publish 

formal papers before completing the Ph.D.? 

B-17^ To what extent do you feel that che faculty and students of the^L£R have 
produced research critical for the field? " 

B-18: What would you view as LER's overall impact on the field of educational 
research? Estimate what LER's impact is likely to be 10 years from now. 

B-19: What do you perceive as the greatest strength of LER. The second greatest 
strength? 



